Natural language understanding (NLU) refers to the technology that allows computers to understand, or derive meaning from, written human languages. In general, NLU systems determine meaning from text. The meaning, and potentially other information extracted from the text, can be provided to other systems. The meaning can drive business logic, effectively trigging some programmatic function corresponding to the meaning. In general, determining meaning from text involves two tasks referred to as “interpretation” and “token extraction.” Interpretation refers to the determination of intent from a text input or the determination of an intended action implied by a text input, e.g., a class. Token extraction refers to the identification of keywords or “tokens” specified in the text input that support, or further elaborate upon, the determined intent.
In some cases interpretation alone is sufficient to resolve the meaning of a text input. For example, a text input such as “How much do I have in my account” specifies an intent or class such as “request account balance.” No tokens are needed to further elaborate upon the request for the account balance. In other cases, tokens are needed to resolve the meaning of the text input. For example, the text input “I'd like to order call forwarding” suggests a request for ordering a service or a product. The particular product desired, however, is not known unless token extraction is applied, in which case, the phrase “call forwarding” can be extracted as a token specifying a particular type of requested service.
Often tokens are extracted using “named entities.” Named entities, effectively, are place holders that map to grammars. In the example above, the phrase “call forwarding” can map to a “service” named entity. The service named entity can map to a grammar specifying the various services that can be understood or recognized, e.g., call waiting, call forwarding, voice mail, or other keywords and/or phrases indicative of a particular service.
In some cases, text inputs include no named entities, making token extraction difficult. For example, the text input “I want to see the name and number of the person calling before I answer the phone,” does not include the phrase “caller ID.” Though the text input refers to caller ID, which would be mapped to, for example, the service named entity, the text input does not explicitly include the phrase “caller ID.”
In other cases, the text input does include a named entity; however, the named entity is used in a different context. The context in which the named entity exists within the sentence is not relevant to the determined intent. For example, in the text input “I am not receiving any phone calls, they go straight to voice mail,” the term “voicemail” would be identified as a named entity. The object of the problem, however, is that the user is not receiving calls. Voicemail is not the object of the problem. Identifying “voicemail” as a named entity in this case can lead to an incorrect resolution of the meaning of the text input.